Papers by Khyathi Raghavi Chandu
A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization (2023.acl-long)
Copied to clipboard
Lining Zhang, Simon Mille, Yufang Hou, Daniel Deutsch, Elizabeth Clark, Yixin Liu, Saad Mahamood, Sebastian Gehrmann, Miruna Clinciu, Khyathi Raghavi Chandu, João Sedoc
| Challenge: | Using crowdsourcing, it is difficult to obtain high-quality annotations for difficult tasks. |
| Approach: | They propose a recruitment pipeline to recruit high-quality Amazon Mechanical Turk workers . they filter out subpar workers before they carry out the evaluations . |
| Outcome: | The proposed method can filter out subpar workers before they carry out evaluations and obtain high-agreement annotations with similar constraints on resources. |
Reading Between the Lines: Exploring Infilling in Visual Narratives (2020.emnlp-main)
Copied to clipboard
| Challenge: | Generating long form narratives from multiple modalities requires a model to learn surrounding contextual information by masking spans of input while decoding attempts in generating the entire text. |
| Approach: | They propose to use infilling techniques to generate textual descriptions from images that are rich in contextual dependencies. |
| Outcome: | The proposed model outperforms existing models in visual storytelling by generating text from a large scale dataset of 46,200 procedures and 340k pairwise images and textual descriptions. |
Denoising Large-Scale Image Captioning from Alt-text Data Using Content Selection Models (2022.coling-1)
Copied to clipboard
| Challenge: | Recent approaches to training large-scale image captioning (IC) models often fall short in terms of performance in leveraging noisy datasets in favor of clean annotations. |
| Approach: | They propose a technique that breaks down the task into two smaller, more controllable tasks - skeleton prediction and skelet-based caption generation. |
| Outcome: | The proposed method can generate better and denoised captions when using noisy datasets. |
Grounding ‘Grounding’ in NLP (2021.findings-acl)
Copied to clipboard
| Challenge: | Cognitive Science defines "grounding" as the process of establishing mutual information between two interlocutors. |
| Approach: | They examine the gaps between NLP and Cognitive Science definitions of "grounding" they propose ways to create new tasks or repurpose existing ones to achieve a more complete sense of grounding . |
| Outcome: | The authors examine the gaps between definitions of grounding and cognitive science . they show that there are ways to improve existing tasks or repurpose existing ones . |
GEMv2: Multilingual NLG Benchmarking in a Single Line of Code (2022.emnlp-demos)
Copied to clipboard
Sebastian Gehrmann, Abhik Bhattacharjee, Abinaya Mahendiran, Alex Wang, Alexandros Papangelis, Aman Madaan, Angelina Mcmillan-major, Anna Shvets, Ashish Upadhyay, Bernd Bohnet, Bingsheng Yao, Bryan Wilie, Chandra Bhagavatula, Chaobin You, Craig Thomson, Cristina Garbacea, Dakuo Wang, Daniel Deutsch, Deyi Xiong, Di Jin, Dimitra Gkatzia, Dragomir Radev, Elizabeth Clark, Esin Durmus, Faisal Ladhak, Filip Ginter, Genta Indra Winata, Hendrik Strobelt, Hiroaki Hayashi, Jekaterina Novikova, Jenna Kanerva, Jenny Chim, Jiawei Zhou, Jordan Clive, Joshua Maynez, João Sedoc, Juraj Juraska, Kaustubh Dhole, Khyathi Raghavi Chandu, Laura Perez Beltrachini, Leonardo F . R. Ribeiro, Lewis Tunstall, Li Zhang, Mahim Pushkarna, Mathias Creutz, Michael White, Mihir Sanjay Kale, Moussa Kamal Eddine, Nico Daheim, Nishant Subramani, Ondrej Dusek, Paul Pu Liang, Pawan Sasanka Ammanamanchi, Qi Zhu, Ratish Puduppully, Reno Kriz, Rifat Shahriyar, Ronald Cardenas, Saad Mahamood, Salomey Osei, Samuel Cahyawijaya, Sanja Štajner, Sebastien Montella, Shailza Jolly, Simon Mille, Tahmid Hasan, Tianhao Shen, Tosin Adewumi, Vikas Raunak, Vipul Raheja, Vitaly Nikolaev, Vivian Tsai, Yacine Jernite, Ying Xu, Yisi Sang, Yixin Liu, Yufang Hou
| Challenge: | Evaluations in machine learning rarely use the latest metrics, datasets, or human evaluation in favor of remaining compatible with prior work. |
| Approach: | They propose to use the Generation, Evaluation, and Metrics Benchmark to integrate new evaluation methods into existing evaluations. |
| Outcome: | The proposed evaluation infrastructure bridges the gap between the advantages of leaderboards and in-depth and evolving evaluations by allowing model developers to benefit from each other's work. |
Switch Point biased Self-Training: Re-purposing Pretrained Models for Code-Switching (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Code-switching (CS) is a phenomenon of switching between multiple languages . current models cannot handle CS due to lack of annotated data and limited resources. |
| Approach: | They propose a self-training method to repurpose existing models using a switch-point bias by leveraging unannotated data to reduce the gap between the switch point performance and retain overall performance on two distinct language pairs. |
| Outcome: | The proposed model reduces the gap between the switch point performance while retaining the overall performance on two distinct language pairs. |